Ground-based data for PRLA

Get the time and weather conditions for all spectra from a site. Start with preferred location (buoys), then can expand (inlet, outlet, reach) where other overlapping data are available.

Updating the AOS and AIS data is handled separately in neon-aqu-aos. AOP flight times are determined from L1 radiance data.

mysite = 'PRLA'

Chlorophyll - AOS

From processed chl a files on Box
reported in ug/L
seasonal sampling, 2 analytes

See how many locations chl data is collected from

See how many locations chl data is collected from (variable Y axis)

Availability of AOS data for each flight date

Default threshold 1 = 30 days
Default threshold 2 = 60 days

flightdate	aos_before	aos_after	days_before	days_after	min_days	meets_thresh1	meets_thresh2
2016-06-29	NA	2019-07-11	NA	1107	1107	FALSE	FALSE
2017-06-21	NA	2019-07-11	NA	750	750	FALSE	FALSE
2017-06-26	NA	2019-07-11	NA	745	745	FALSE	FALSE
2019-07-26	2019-07-11	2019-09-10	15	46	15	TRUE	TRUE
2019-07-27	2019-07-11	2019-09-10	16	45	16	TRUE	TRUE
2019-07-30	2019-07-11	2019-09-10	19	42	19	TRUE	TRUE
2020-06-24	2020-06-15	2020-09-29	9	97	9	TRUE	TRUE
2020-06-26	2020-06-15	2020-09-29	11	95	11	TRUE	TRUE
2020-07-02	2020-06-15	2020-09-29	17	89	17	TRUE	TRUE

flightdate	aos_match	days
2019-07-26	2019-07-11	15
2019-07-27	2019-07-11	16
2019-07-30	2019-07-11	19
2020-06-24	2020-06-15	9
2020-06-26	2020-06-15	11
2020-07-02	2020-06-15	17

Make sure plant algae lab units are all micrograms per liter
Check sample conditions

## [1] "microgramsPerLiter"

## [1] "condition ok"

These are the closest chl AOS values for each flight date

aos_match	flightdate	days	pheophytin	chlorophyll a
2019-07-11	2019-07-26	15	3.01	2.31
2019-07-11	2019-07-27	16	3.01	2.31
2019-07-11	2019-07-30	19	3.01	2.31
2020-06-15	2020-06-24	9	4.20	3.45
2020-06-15	2020-06-26	11	4.20	3.45
2020-06-15	2020-07-02	17	4.20	3.45

Chlorophyll - Algal Community

TO DO!!!!!

Relative abundance by division, class for diatom slides and soft algae count
Top 10 morphospecies for each method?

Chlorophyll - AIS

Sensor chl data availability

flightline_datetime	check_30day	check_10day	check_1day	check_12hr
2016-06-29 15:45:47	FALSE	FALSE	FALSE	FALSE
2016-06-29 15:52:09	FALSE	FALSE	FALSE	FALSE
2016-06-29 17:22:24	FALSE	FALSE	FALSE	FALSE
2017-06-21 20:58:28	FALSE	FALSE	FALSE	FALSE
2017-06-26 15:33:21	FALSE	FALSE	FALSE	FALSE
2017-06-26 16:04:51	FALSE	FALSE	FALSE	FALSE
2017-06-26 16:11:11	FALSE	FALSE	FALSE	FALSE
2019-07-26 15:20:23	TRUE	TRUE	TRUE	TRUE
2019-07-26 16:22:11	TRUE	TRUE	TRUE	TRUE
2019-07-26 22:09:25	TRUE	TRUE	TRUE	TRUE
2019-07-27 15:27:04	TRUE	TRUE	TRUE	TRUE
2019-07-30 15:34:20	TRUE	TRUE	TRUE	TRUE
2020-06-24 17:36:27	TRUE	TRUE	TRUE	TRUE
2020-06-24 16:26:15	TRUE	TRUE	TRUE	TRUE
2020-06-24 16:18:44	TRUE	TRUE	TRUE	TRUE
2020-06-26 19:28:06	TRUE	TRUE	TRUE	FALSE
2020-07-02 17:14:10	TRUE	TRUE	FALSE	FALSE

Chlorophyll AIS plots 2019

Filtered to only 0 quality flags

Chlorophyll AIS plots 2020

No QF filters on these plots
Is QF because of lack of calibration?

Chlorophyll AIS vs AOS regression

TO DO!!!!!

DOM - AOS

From processed Surface water chemistry files on Box

Ignoring data before 2016

## Warning: Removed 2 rows containing missing values (geom_point).

Correct older UV Absorbance wavelengths from 250 to 254, assuming all 250 nm should be 254.

swchem_site_df <- swchem_site_df %>%
  dplyr::mutate(analyte = dplyr::case_when(analyte == 'UV Absorbance (250 nm)' ~
                                             'UV Absorbance (254 nm)', TRUE ~ analyte))

## Warning: Removed 2 rows containing missing values (geom_point).

How much does SUVA 254/280 ratio change?

How much does organic carbon dissolved proportion change?

Adjusts sample IDs in a new column to group together raw and filtered, while keeping track of duplicates

## Warning: Removed 11 rows containing missing values (geom_point).

Match DOM AOS and AOP data

flightdate	aos_before	aos_after	days_before	days_after	min_days	meets_thresh1	meets_thresh2
2016-06-29	2014-06-19	2016-10-19	741	112	112	FALSE	FALSE
2017-06-21	2017-06-08	2017-07-06	13	15	13	TRUE	TRUE
2017-06-26	2017-06-08	2017-07-06	18	10	10	TRUE	TRUE
2019-07-26	2019-07-01	2019-08-06	25	11	11	TRUE	TRUE
2019-07-27	2019-07-01	2019-08-06	26	10	10	TRUE	TRUE
2019-07-30	2019-07-01	2019-08-06	29	7	7	TRUE	TRUE
2020-06-24	2020-06-09	2020-06-30	15	6	6	TRUE	TRUE
2020-06-26	2020-06-09	2020-06-30	17	4	4	TRUE	TRUE
2020-07-02	2020-06-30	2020-08-04	2	33	2	TRUE	TRUE

flightdate	aos_match	days
2017-06-21	2017-06-08	13
2017-06-26	2017-07-06	10
2019-07-26	2019-08-06	11
2019-07-27	2019-08-06	10
2019-07-30	2019-08-06	7
2020-06-24	2020-06-30	6
2020-06-26	2020-06-30	4
2020-07-02	2020-06-30	2

These are the closest CDOM-related AOS values for each flight date

A list of values means there are duplicate samples (multiple sampleIDs) from same date.
TOC is reported in milligramsPerLiter
Fe is reported in milligramsPerLiter
DOC is reported in milligramsPerLiter
UV abosrbance values are unitless and specific absorbance, multiply by DOC for actual absorbance

aos_match	flightdate	days	UV Absorbance (254 nm)	TOC	Fe	DOC	UV Absorbance (280 nm)
2017-06-08	2017-06-21	13	0.5578	34.12	5e-04	29.97	0.2876
2017-07-06	2017-06-26	10	0.5566, 0.5710, 0.5653	37.87, 38.11, 38.82	5e-04, 5e-04, 5e-04	31.68, 31.81, 32.11	0.2887, 0.2961, 0.2950
2019-08-06	2019-07-26	11	0.7538	36.05	0.01	35.01	0.3852
2019-08-06	2019-07-27	10	0.7538	36.05	0.01	35.01	0.3852
2019-08-06	2019-07-30	7	0.7538	36.05	0.01	35.01	0.3852
2020-06-30	2020-06-24	6	NULL	26.05, 26.49, 26.18	NULL	NULL	NULL
2020-06-30	2020-06-26	4	NULL	26.05, 26.49, 26.18	NULL	NULL	NULL
2020-06-30	2020-07-02	2	NULL	26.05, 26.49, 26.18	NULL	NULL	NULL

DOM - AIS

availability of processed fDOM time series within different time windows

flightline_datetime	check_30day	check_10day	check_1day	check_12hr
2016-06-29 15:45:47	FALSE	FALSE	FALSE	FALSE
2016-06-29 15:52:09	FALSE	FALSE	FALSE	FALSE
2016-06-29 17:22:24	FALSE	FALSE	FALSE	FALSE
2017-06-21 20:58:28	FALSE	FALSE	FALSE	FALSE
2017-06-26 15:33:21	FALSE	FALSE	FALSE	FALSE
2017-06-26 16:04:51	FALSE	FALSE	FALSE	FALSE
2017-06-26 16:11:11	FALSE	FALSE	FALSE	FALSE
2019-07-26 15:20:23	TRUE	TRUE	TRUE	TRUE
2019-07-26 16:22:11	TRUE	TRUE	TRUE	TRUE
2019-07-26 22:09:25	TRUE	TRUE	TRUE	TRUE
2019-07-27 15:27:04	TRUE	TRUE	TRUE	TRUE
2019-07-30 15:34:20	TRUE	TRUE	TRUE	TRUE
2020-06-24 17:36:27	TRUE	TRUE	TRUE	TRUE
2020-06-24 16:26:15	TRUE	TRUE	TRUE	TRUE
2020-06-24 16:18:44	TRUE	TRUE	TRUE	TRUE
2020-06-26 19:28:06	TRUE	TRUE	TRUE	FALSE
2020-07-02 17:14:10	TRUE	TRUE	FALSE	FALSE

fDOM plots 2019

## Warning: Removed 3170 rows containing missing values (geom_point).

## Warning: Removed 21 rows containing missing values (geom_point).

## `geom_smooth()` using method = 'loess' and formula 'y ~ x'

## `geom_smooth()` using method = 'loess' and formula 'y ~ x'

## `geom_smooth()` using method = 'loess' and formula 'y ~ x'

fDOM plots 2020

Not filtered on QFs
QF because of lack of calibration?

## Warning: Removed 8658 rows containing missing values (geom_point).

## Warning: Removed 2552 rows containing missing values (geom_point).

## Warning: Removed 444 row(s) containing missing values (geom_path).

## Warning: Removed 444 rows containing missing values (geom_point).

## Warning: Removed 4737 rows containing missing values (geom_point).

fDOM AIS vs DOC AOS regression

TO DO!!!!

Suspended Matter - AOS

From processed Surface water chemistry files on Box
TDS passes through .45 micron filter
TSS is from weight of dried mass that doesnt pass through filter

Data from all locations at site

## geom_path: Each group consists of only one observation. Do you need to adjust
## the group aesthetic?
## geom_path: Each group consists of only one observation. Do you need to adjust
## the group aesthetic?
## geom_path: Each group consists of only one observation. Do you need to adjust
## the group aesthetic?

sm_site_df <- sm_site_df %>%
  dplyr::filter(namedLocation %in% my_loc)

## Warning: Transformation introduced infinite values in continuous y-axis

## Warning: Transformation introduced infinite values in continuous y-axis

How much does TDS/TSS change?

Adjusts sample IDs in a new column to group together raw and filtered, while keeping track of duplicates
Dissolved passes through .45 micron filter, suspended does not

## Warning: Removed 1 row(s) containing missing values (geom_path).

## Warning: Removed 2 rows containing missing values (geom_point).

Match sediment aos to aop dates

source('R/match-aop-aos.R')

sm_match_list <- match_aop_aos(mysite_dates, sm_site_df, 'collect_date')

knitr::kable(sm_match_list$dates)

flightdate	aos_before	aos_after	days_before	days_after	min_days	meets_thresh1	meets_thresh2
2016-06-29	2014-06-19	2016-10-19	741	112	112	FALSE	FALSE
2017-06-21	2017-06-08	2017-07-06	13	15	13	TRUE	TRUE
2017-06-26	2017-06-08	2017-07-06	18	10	10	TRUE	TRUE
2019-07-26	2019-07-01	2019-08-06	25	11	11	TRUE	TRUE
2019-07-27	2019-07-01	2019-08-06	26	10	10	TRUE	TRUE
2019-07-30	2019-07-01	2019-08-06	29	7	7	TRUE	TRUE
2020-06-24	2019-12-04	2021-05-04	203	314	203	FALSE	FALSE
2020-06-26	2019-12-04	2021-05-04	205	312	205	FALSE	FALSE
2020-07-02	2019-12-04	2021-05-04	211	306	211	FALSE	FALSE

knitr::kable(sm_match_list$matches)

flightdate	aos_match	days
2017-06-21	2017-06-08	13
2017-06-26	2017-07-06	10
2019-07-26	2019-08-06	11
2019-07-27	2019-08-06	10
2019-07-30	2019-08-06	7

List of values means there are duplicate samples (multiple sampleIDs) from same date.

aos_match	flightdate	days	TDS	TSS	TSS - Dry Mass
2017-06-08	2017-06-21	13	910.1	32	800
2017-07-06	2017-06-26	10	885.62, 887.64, 891.23	10, 9, 9	900, 900, 1000
2019-08-06	2019-07-26	11	744.6	10	500
2019-08-06	2019-07-27	10	744.6	10	500
2019-08-06	2019-07-30	7	744.6	10	500

Suspended Matter - AIS

processed turbidity water quality time series on box

flightline_datetime	check_30day	check_10day	check_1day	check_12hr
2016-06-29 15:45:47	FALSE	FALSE	FALSE	FALSE
2016-06-29 15:52:09	FALSE	FALSE	FALSE	FALSE
2016-06-29 17:22:24	FALSE	FALSE	FALSE	FALSE
2017-06-21 20:58:28	FALSE	FALSE	FALSE	FALSE
2017-06-26 15:33:21	FALSE	FALSE	FALSE	FALSE
2017-06-26 16:04:51	FALSE	FALSE	FALSE	FALSE
2017-06-26 16:11:11	FALSE	FALSE	FALSE	FALSE
2019-07-26 15:20:23	TRUE	TRUE	TRUE	TRUE
2019-07-26 16:22:11	TRUE	TRUE	TRUE	TRUE
2019-07-26 22:09:25	TRUE	TRUE	TRUE	TRUE
2019-07-27 15:27:04	TRUE	TRUE	TRUE	TRUE
2019-07-30 15:34:20	TRUE	TRUE	TRUE	TRUE
2020-06-24 17:36:27	TRUE	TRUE	TRUE	TRUE
2020-06-24 16:26:15	TRUE	TRUE	TRUE	TRUE
2020-06-24 16:18:44	TRUE	TRUE	TRUE	TRUE
2020-06-26 19:28:06	TRUE	TRUE	TRUE	FALSE
2020-07-02 17:14:10	TRUE	TRUE	FALSE	FALSE

Turbidity plots 2019

filtered to no quality flags

Turbidity plots 2020

Not filtered for quality flags
QF because of lack of calibration?

## Warning: Removed 8537 row(s) containing missing values (geom_path).

## Warning: Removed 8658 rows containing missing values (geom_point).

## Warning: Removed 2438 row(s) containing missing values (geom_path).

## Warning: Removed 2552 rows containing missing values (geom_point).

## Warning: Removed 444 row(s) containing missing values (geom_path).

## Warning: Removed 444 rows containing missing values (geom_point).

## Warning: Removed 4737 row(s) containing missing values (geom_path).

## Warning: Removed 4737 rows containing missing values (geom_point).

Turbidity AIS vs TDS and TSS AOS regression